Search Results: "Aigars Mahinovs"

7 January 2011

Paul Wise: Another year, another log entry

It has been almost a full year since my last log entry. It has been a busy work year, I attended some nice conferences and did minimal FLOSS stuff. On the work side of things I was a third of an Australian VoIP startup that came and went. I setup Debian servers, installed OpenSIPS and associated software, wrote OpenSIPS scripts, wrote peripheral software and did customer support. We had a good thing going there for a while, some fans on the Whirlpool forums but in the end there wasn't enough money for the requisite marketing and local market circumstances were squeezing Australian VoIP providers anyway. On the conference side of things I went to LCA 2010, the Thai Mini-DebCamp 2010, DebConf10 and FOSSASIA 2010. Had a great time at all of them. At LCA 2010 in windy Wellington, New Zealand the distributions summit organised by Martin Krafft was one of the highlights. It was dominated by Debian/Ubuntu talks but there were some other interesting ones, especially the one on GoboLinux's integration of domain-specific package managers. Also excellent were the keynotes given by Gabriella Coleman (Best & worst of times), Mako Hill (Antifeatures) and others, which I felt gave LCA an improved and very welcome focus on software freedom. There were quite a few Debian folks at LCA, it was great to hang out with them during the week and afterwards. Monopedal sumo with mako and others was hilarious fun. At the Thailand Mini-DebCamp 2010 in Khon Kaen, I was glad to see Andrew Lee (Taiwan) and Christian Perrier (France) again and meet Yukiharu YABUKI (Japan) and Daiki Ueno (Japan). In addition to the five international folks, there were quite a few locals, including Thailand's currently sole Debian member, Theppitak Karoonboonyanan. The event was hosted at Khon Kaen University and opened with my talk about the Debian Social Contract and the Debian Free Software Guidelines. This was followed by a number of talks about Debian package building, a 3-day BSP where we touched 57 bugs, a great day of sightseeing and talks about i18n, derivative distros, keysigning, mirrors, contribution and a discussion about DebConf. During the week there was also the usual beersigning, combined with eating of unfamiliar and "interesting" Thai snacks. After the conference Andrew and I roamed some markets in Bangkok and got Thai massages. Beforehand I also visited a friend from my travels on the RV Heraclitus in Chiang Mai, once again experiencing the awesomeness of trains in Asia, unfortunately during the dry season this time. I took a lot of photos during my time in Thailand and ate a lot of great and spicy food. As a vegetarian I especially appreciated the organiser's efforts to accommodate this during the conference. At DebConf10 in New York City, by far the highlight was Eben Moglen's vision of the FreedomBox. Negotiating the hot rickety subways was fun, the party at the NYC Resistor space was most excellent, Coney Island was hot and the water a bit yuck, zack threw a ball, the food and campus was really nice. Really enjoyed the lintian BoF, ARM discussions, shy folks, GPLv3 question time, paulproteus' comments & insights, wiki BoF, puppet BoF, derivatives BoF, Sita, astronomy rooftop, cheese, virt BoF, Libravatar, DebConf11, Brave new Multimedia World, bagels for breakfast, CUT, OpenStreetMap & lightning talks. Having my power supply die was not fun at all. Afterwards I hung out with a couple of the exhausted organisers, ate awesome vegan food and fell asleep watching a movie about dreams. One weird thing about DebConf10 was that relatively few folks used the DebConf gallery to host their photos, months later only myself and Aigars Mahinovs posted any photos there. At FOSSASIA 2010 in H Ch Minh City (HCMC) was a mini-DebConf. I arrived at the HCMC airport and was greeted by Huyen (thanks!!), one of FOSSASIA's numerous volunteers, who bundled me into a taxi bound for the speakers accommodation and pre-event meetup at The Spotted Cow Bar. The next day the conference opened at the Raffles International College and after looking at the schedule I noticed that I was to give a talk about Debian that day. Since I didn't volunteer for such a talk and had nothing prepared, the schedule took me by surprise. So shortly after an awesome lunch of Vietnamese pancakes we gathered some Debian folks and a random Fedora dude and prepared a short intro to Debian. The rest of the day the highlights were the intro, video greetings and the fonts, YaCy and HTML5 talks. The next day the Debian MiniConf began with Arne Goetje and everyone trying to get Debian Live LXDE USB keys booted on as many machines in the classroom as possible (many didn't boot). Once people started showing up we kicked off with Thomas Goirand's introduction to the breadth of Debian. Others talked about Debian pure blends, Gnuk and building community and packages. The second last session was about showing the Vietnamese folks in the room how to do l10n and translation since Debian had only one Vietnamese translator (Clytie Siddall). After manually switching keyboard layouts (seems LXDE doesn't have a GUI for that) on the English LXDE installs, the two Cambodian folks were able to do some Khmer translation too. This was a great session and it resulted in two extra Vietnamese translators joining Debian. It went over time so I didn't end up doing my presentation about package reviewing. We rushed off to a university where the random Fedora ch^Wambassador was hosting a Fedora 14 release party in a huge packed classroom. There were a lot of excited faces, interesting and advanced questions and it was in general a success. Afterwards we had some food, joined up with some other speakers and ended up in a bar in the gross tourist zone. On the final day we hung around in the Debian room, went downstairs for the group photo and final goodbyes. Later we found a place with baked goods, coffee and juices and navigated the crazy traffic to a nice local restaurant. The next morning Arne & I went to the airport, others went on a Mekong Delta tour and Jonas hung out with the organisers. I took less photos than at other events but got a few interesting ones. I avoided doing a lot of FLOSS stuff over the last year, I hope to work on some things in the coming months; I'm also planning some interesting travel and acquiring some new technological goods, more on those in some later posts.

6 January 2011

Enrico Zini: Streaming JSON objects with python

Streaming JSON objects with python Dear Lazyweb, Suppose I'd like to create an HTTP service that gives you a looong list of JSON objects. For example, one JSON object for every package in Debian. One way I could do that is to just transfer a big, dozens of megabytes JSON array with one element per package. With the standard json module in python, this requires the client to buffer all the data in memory for decoding, as the JSON result would be something like this:
[
    info for package 1  ,
    info for package 2  ,
  ...dozens of megabytes...
    info for package 30000  ,
  ...
]
However, it would be reasonable to engineer things so that the client can process packages one at a time, as they are produced. This can dramatically reduce client memory usage, as well as remove a large latency between the start of the request and the start of processing. One trivial idea would be to just send JSON objects one after the other, and sort of call load multiple times. It doesn't work:
import json
from cStringIO import StringIO
a = StringIO()
json.dump( 1:3 , a)
json.dump( 2:4 , a)
a.reset()
print a.getvalue()
print json.load(a)
gives:
$ python z1.py 
 "1": 3 "2": 4 
Traceback (most recent call last):
  File "z1.py", line 8, in <module>
    print json.load(a)
  File "/usr/lib/python2.6/json/__init__.py", line 267, in load
    parse_constant=parse_constant, **kw)
  File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.6/json/decoder.py", line 322, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 8 - line 1 column 16 (char 8 - 16)
And the json module documentation does not provide any way of telling the "stream" decoder to just stop parsing after the first valid object. An alternative can be to use chunked transfer encoding and transfer each JSON object in a separate HTTP chunk. It would add the overhead of a little HTTP header per JSON object, which could be significant when generating lots of small JSON objects, but it would be an implementation that strictly follows HTTP and JSON standards to the letter in a clean way. Except:
  1. How do you stream HTTP chunks in WSGI? The WSGI specification's only mention of chunks is that if you produce a result as a generator of "chunks", they are just sent one after the other (effectively, concatenated) in the response stream.
  2. How do you read and process one chunk at a time with urllib/urllib2? I searched on Google but I only found people in tears.
Another alternative is to wrap the JSON objects in some custom container protocol, but then we are not talking standards anymore. Is this all you can do with the standard python library? Can this only be done by adding a dependency on a C library with two obscure separately maintained bindings, or by writing my own JSON parser? Ideas? Please mail enrico@enricozini.org and I'll add them here. Updates Wouter van Heyst mentions that he heard of ijson, which indeed has my exact use case in the synopsis, but after some research looks like a third binding for yajl. Aigars Mahinovs suggests using YAML instead of JSON. Indeed YAML supports streaming very well (it's even already supported in DDE, but last time I tried to do anything serious with it, PyYaml was slooow. Of course, this being the Python ecosystem, There Are Lots Other Ways To Do It And You Shall Be Patronised For Not Chosing The Proper One, but Debian only has PyYaml. Another itch with YAML is that there is no YAML parser in the standard python library, which requires clients to add an extra dependency. Jyrki Pulliainen has so far mailed my favourite solution: disable formatting in the encoder and separate JSON objects with newlines:
we built a streaming JSON app as a part of our platform. However, we
didn't stream a single JSON object as that was a bit cumbersome. When
having syntax like:
[
  <json-obj1>,
  <json-obj2>,
  ....
  <json-objN>
]
The reader has some problems. Should it read the whole stream to find
the closing bracket ] even if the stream would me tens of megabytes?
Should it try to split on , and parse single objects? What if the
objects contain , so should there be some regexp magic?
We ended up streaming separate JSON objects separated by newlines
(\n). This way the reader can read until hitting \n or end of stream
and then split the result on newlines. The new stream looked like
this:
<json-obj1>\n
<json-obj2>\n
...
<json-objN>\n
This way we could efficiently stream hundreds of thousands of JSON
objects from server to client without reserving a load of memory. For
the WSGI part how to do this, the best would probably to do a
generator that would yield a single JSON object with the newline (we
didn't use WSGI back then). Reading can be done using any reader
capable of handling streams. We used pycurl for reading, but I can't
really recommend it as it is way too complicated for a simple task
like this :p
Hopefully this helps!
This means that if the client has no access to a JSON reader that supports a stream of concatenated toplevel objects, they can still trivially code it on top of any JSON library. This means that working clients can be written based just on Squeeze's python standard library. That's what I'm going to do; I'll probably implement it as a separate 'jsons' output format for DDE, because producing a JSON response with multiple toplevel obejcts somehow breaks the JSON standard. Thanks to all who provided this excellent information!

18 August 2010

Aigars Mahinovs: Debconf10 group photo

Please add your name and names of people you recognise here. Update: 14 people left! The rest of my DebConf10 photos are in my DebConf10 Flickr set. There will still be a few photos from today uploaded to that set tomorrow morning, but it is almost complete. Hint: it also contains a video of DPL pitch and video of MakerBot working. Update: All photos are up and labelled now. Enjoy!

Aigars Mahinovs: Social distribution

I have just looked at traffic to my photos from Debconf10 during and after the event. During the event I sent out the link to the photos to Twitter/Identi.ca and to IRC channel of the conference. The amount of people visiting my photos rose from <100 per day to around 1000 per day. That might be the number of people that follow Debian actively enough to either follow the conference directly or look at #debian or #debconf or #debconf10 Twitter hashtags (if only half of people care about photos, multiply that number by 2).
Now right after I posted the group photo to my blog and to Debian Planet on the 7th of August, the number of daily views sharply rose to 10 000 per day and stayed at that number for 3 days, then decayed to 5k, 3k, 2k, 1k. We can draw two conclusions from this:
  1. Debian Planet is far more popular than Debconf info or Debian-related hashtags on Twitter/Identi.ca. It could be estimated that the Debian Planet is 50-100 times more popular.
  2. Readers of Debian Planet read the posted articles with a delay. It looks like most will have read an article after 6-7 days, but there might also be a tail, I will kee an eye on that over the next weeks
I thought it might be interesting to people :) Note: Only 14 blank spots left in the group photo faces page! Let s get the hunt on and finish it!

31 July 2010

Aigars Mahinovs: Debcamp 10 the early days

After I finished an re-read my epic post describing my first day and a half at Debconf10 I suddenly realised that if I continue to describe the rest of 13 days here in such style and verbosity this would become a book, a boring one at that, so I decided to limit myself to 2 posts for Debcamp and then a post a day for the Debconf. But about the first half of the Debcamp week there are things that are as usual and there are things that are not. Let s start with the usual bits: more people arriving every day, evening parties are getting more and more fun and wild (or so people say) and the Internet and power at the hacklabs magically becomes more and more reliable day by day, the cabling mess grows organically trying to spread out evenly and not overload any individual socket, extension cords with non-local plugs appear and spread the load some more, people stress out, volunteer, crash out and repeat. All in all nothing can stop the freight train of few hundred Debian developers determined to have fun. With out own DFSG-free definition of fun. (But this year we appear to be running very close to the edge on volunteers if we run out of those, there will be a train-wreck, so please volunteer!) Now off to the unusual stuff. The City is well it is cool and very, very impressive. It is not terribly different to me at least I am used to such structure of the city: grid based layout of the downtown city (with some parks) and then branching out to separate, but different across-the-river districts and then sprawling out to slightly chaotic suburbs. Riga is very much like New York in the design. The difference is that New York is 10-20 times larger: the areas are 10 times larger, the buildings are 3-4 times higher on average, the streets are 2-3 times wider and yet have more traffic and there are far, far more shops, shows and restaurants. Also there is the subway. It is a great thing in that it hides the size of the city take a bus downtown once, to find out what I mean it takes ages to drive the distance that takes 20 minutes on the subway. I like this city, I am getting a feeling, that I could live here without much problem for me. So far I ve only had this feeling in Riga and Berlin. On the other hand this is USA. I ve so far only encountered one bad thing about it, but it is a pretty big one food. Basically all food that I ve tried so far in the USA has been crap. It was tasty salt, sugars and fat took care on my brain thinking it s food, but underneath that it was pretty crap: there was no texture, no content, no soul. The John Jay cafeteria where we are eating lunch and dinner is better than the most other options, because it is an all-you-can-eat buffet where you can choose your poison and it was also the place where I had the best piece of food in USA so far a slice of pepperoni pizza, that had a bit of taste behind the fat. Also, surprisingly, the fast food options (McDonalds, Burger King, ) are better than the equivalents in Europe, far better. In fact the fast food is cheap and in some cases tastes better than regular food, so there is no wonder why people might prefer it in some situations. So it is easy to see how people just accept shoving bland fast food or bland all-you-can-eat food into their mouths and not think much about it and thus become fat and put undue stress on their health. I expect the average weight of Debian developers to increase by 5 kg by the end of the conference. If you want to prove me wrong, running with bubulle would help you a lot with that. Events downtown. New York is a huge city with a lot of events going on every day, so everyone should be able to find something of interest for them. For me it was the free tickets for tapeings of The Daily Show and The Colbert Report. I got into both of those and for me The Colbert Report was by far the better experience. For one you can not get into Daily Show without a ticket (some tickets show up on the web site on the morning of the shoot around 11am), while 18 people from the stand-by list at Colbert Report got in. At The Daily Show you stand in a live line with ticket holders in one line and stand-by line separate. If you have tickets and arrive around 3pm you should be fine any later than that and you risk to be standing outside even with a ticket. You find that out around 4.30pm when they hand out physical tickets you have one, you get in. If not you might rush over to Colbert Report where a more humane system is used a staffer takes down your name and email on a numbered list and then you can walk around until 5.30pm then they let the ticket holders in and after that call the names of stand-by people that get in. The Colbert Report studio is brighter, more colourful and closer to the action, also it is very rare that cameras block the view from the audience (which is common on The Daily Show). It might have also been my luck when I got on the most boring The Daily Show episode I can remember, ever. The best joke was a woman asking John is she could get a ticket for her friend in August. The warm-up act and crowd control at The Colbert Report was also way better: Colbert staff was hyper, security was ever-present, warm-up was funnier and very engaging (he grilled me for several minutes and I replied making the audience laugh very hard explaining that Latvia was like India of Eastern Europe in regards to IT exports) and Colbert himself was very gracious talking to us out-of-character before the show for a good 10 minutes. Location, location, location. The talk rooms and event rooms and hacklabs are spread out across multiple buildings and there is a lot of other activities going on in those buildings besides Debconf, so moving between place might be confusing for the first day or two. Also the rooms are quite dark that might cause a noise problem for me and the video team as we ll have to up the sensitivity settings on our gear. The video team is working on fixing that by throwing a bit more light on to the speakers. Elevators are wicked fast, but can also be confusing, because the ground floor is on a different number for different buildings it can be G, 1 or even 4. Usually on campus there is a star next to the level with the exit. The dorms. I am in the Carman building and it looks like its interior has not been updated since it was built large ceramic bricks with very visible gaps (for the interior walls), raise-to-open windows, huge aircon fans that take up the whole bottom of the window, plumbing from the 60ies (at least). The security is weird on one hand you have to give your room card to the guard when entering the building, but on the other hand the room cleaning crew can simply forget to close your door after they are done. Like this Thursday I returned to the dorm just before lunch only to find the door of my room open. No one was around and nothing was missing, but it s still worrying. In any case I am steadily uploading photos from Debconf 10 to my Flickr page and new stuff should show up every day.

28 July 2010

Aigars Mahinovs: Debconf 10 arrival and first days

I had planned long for this Debconf and with the experience from the previous times I did all I could to reach one goal minimise stress. I think I got it right this time. First of all I got my company (Accenture) pay for the whole thing, so I did not have to worry about sponsorship queue (but instead I must gather and deliver them back some knowledge gained from the conference should be easy). I also managed to get a direct flight there is a weekly flight Tashkent-Riga-New York on Sundays and I got a ticket for that both ways. This took many hours off the travel time and reduced the stress a ton. Then I also did all the prudent travelling things: mostly reading the New York page on Wikitravel. This told me (among other things) what metro line to take to get the best view of the city. And for luggage I took a backpack (for the laptop and walking around) and a roller (for the clothes). I learned a lot from the Up in the air movie, it really teaches one how to take stress out of travel. This went very well in the beginning I took a bus to the airport early, checked in, breezed trough security and then waited and waited. The boarding only started half an hour after we were scheduled to depart. But I can not complain, because the flight attendant called my name (among a few others) just before boarding and changed my ticket I was bumped up to business class :) . It has been many years since I ve flown a regular (non-budget) airline and now first time ever in the business class! We got regular class meals, but at least we got to sit in the big chairs with lots of leg room. It was great! I bought a paper book (from Charles Stross) to read during the flight, because that is the only thing you can do during take-off and landing and I had some stuff to listen to in my iPhone. I tried sleeping and I think I managed to kill a couple hours that way too. It sure helped combat jet lag later on. When I arrived, I had my immigration and customs forms filled out. Latvia is on the USA Visa Waiver program, so I just had to fill out a form online prior to the trip to get a permission to enter USA. I only had one printout of the Waiver confirmation page and my airline took that before giving me a boarding pass, so I had no paper copy to give to the guy at the USA border, but he did not even ask for it he only took the two forms I filled out, scanned my passport, took my fingerprints (all fingers) and a photo. Then he gave me back the customs for (stamped) and I could go get my luggage. After that it was just a matter of walking trough customs and giving the customs official the pre-filled and stamped customs form. I was not stopped further. The AirTrain was a bit confusing you just get on the AirTrain and go where you need to go, you actually pay at the place where AirTrain connects to metro, so when you are coming from the airport you are actually paying ($5) when you are exiting the AirTrain system. There are plenty of AirTrain people that will give you an AirTrain ticked for $5 cash right there or you can use a machine to get it (where you can also pay with a card). The machines also take AmEx, but you must know the PIN code, if your don t know the PIN of you AmEx card (like I don t know the PIN of my corporate AmEx) you can only buy MetroCards at news stands, but there you will not find the unlimited ride tickets only 5 or 10 ride tickets are available. Next I screwed up a bit and thus got a bit lost in the New York metro system. The Wikitravel recommends to take the AirTrain to Jamaica station and from there take J or Z line into the city for the best views. I went to Jamaica and got into the first metro train that came on to the platform. Unfortunately, that was the E line train and it does not cross with J or Z lines after that, so I was stuck. To complicate the situation E line is under repair, so after a few stops the trains diverted to the F line and I had to scramble to figure out where to get out to get to the 1 line where Columbia University is. I ended up spending more than an hour in the metro, but I got there in the end. So I emerge from the 116th street metro station directly in front of the campus entrance gate and it starts raining. A thunderstorm swept across New York as soon as I stepped outside the metro station, so I took shelter a the side of the building. It kept raining. After a few minutes decided that I don t want to wait any more, so I put all my electronics into my backpack and went towards the dorm where my room was. The rain was warm, people around me were playing and running around in the wet grass, it was fun. The guard showed me where to go to check in. It was really simple sign here, here is you magnetic stripe key, have fun. And the room was just as spartan bed, desk, chair, closet, bathroom and a large fan under the window. The first night I did not know that I could actually turn that fan down, so I slept to the sound of full fan blasting, much like on the airplane. The dorm looks ancient it looks like it was build in the 60s and never changed since then. I mean, I ve lived in worse places, but people would at least change wall sockets or faucets or ceiling paint every other decade or so. The newest things here were the magnetic locks on the doors and a TV and a mini fridge in the hallway. After dropping my things I went off to the Mudd building across the campus to check in with people in the hacklab. It was just starting to take shape, but many great and familiar people were already there. It was like walking into a family home same warm welcome smiles and that great feel of belonging to the group. After I quick chat I went off downtown. The plan was that I left my old Canon 400D camera back at home and that I would buy a new Canon 550D here in New York on arrival. Unfortunately I underestimated the travel time on subway and by the time I got back downtown most shops where already closed. I got to checking out a Best Buy and was told there that they don t have 550D in stock. Next morning I went for a full scale search B&H Photo Video store, J&R Express, another BestBuy none of them had the Canon 550D (aka T2i) in stock in a body-only configuration. BestBuy had a few sets with the kit lens, but I did not want paying 100$ extra for something I already have. The last option was Adorama and I got lucky I nabbed the last 550D they had and it was body-only. Now I was back, with a camera in my hands and the Debconf10 could really begin!

8 July 2010

Aigars Mahinovs: Phoning it in

Currently I own and use an iPhone 3G. I bought it almost two years ago, when the local phone provider LMT started offering the iPhone legally. I had a pretty good experience with it most of the time, but now it is showing its age: I will try to send my iPhone in for repair hoping to extend its usefulness, but frankly it looks that I might have to get a new phone at or before Debconf. So I am considering my options for a new phone and so far I have 3 main options each with some sub-options. The main options are: iPhone 4, MeeGo and Android. Each choice has its benefits and drawbacks and each also has several sub-options related either to specific phone models or to purchase methods. What I need is a phone with: calling function, contact information synchronised to Google, support for Google Mail/Contacts/Calendar simultaneously with one Exchange account also providing mail/contacts/calendars, music, audiobooks saving my place in the audio book and in the list of books, fully featured Twitter client with Twitpic and location support, encrypted password storage application (preferably open source), GPS with maps, ability to download maps for offline use, Geocaching support with offline cache info and logging, Skype client, ability to write in Latvian, English and Russian in all applications, reliable backup and restore of phones data including settings and data of installed apps. Optional features: last.fm streaming and reporting support, ebook applications with purchasable ebooks, high quality video and photo recording (for a phone), ability for me to write applications on the phone, ssh client, wireless data upload to the phone, ability to change the phones battery without loosing data in the phone (on the go), compass, physical keyboard, confidence in getting software upgrades and developer interest for the next 2 years. I ll try to describe all of the options here to organise my thoughts and maybe also help someone else make the choice. Option 1: iPhone 4 To get the iPhone 4 I have two options: LMT and unlocked. The official arrival of iPhone 4 to Latvia is expected in September and it will likely be sold by LMT for the same prices as iPhone 3GS is now. If I choose this scenario, I will need to survive with my semi-broken iPhone for 2-3 more months. Total price over 2 years (LVL): 131 (purchase price) + 24 * 8 (surcharge) = 323 LVL. Note: the surcharge is the extra cost of the iPhone call plan compared to a comparable regular plan (Vien dais 7 + Internets telefon 5). If I go and purchase an unlocked iPhone 4 in a country where such units are sold (UK is the most easily accessible choice, because a few of my workmates travel weekly to and from UK) then the total price over 2 years would be the purchase price: 499 GBP + 65 GBP (2 year warranty extension) = 505 LVL Now lets go over the general benefits and downsides of the iPhone 4, first the positives: Drawbacks: Option 2: Maemo/MeeGo Some of my colleagues at work (in the Riga office of Accenture) have N900 phones and I have been exposed to people with raw enthusiasm towards the Maemo platform ever since the Debconf in Helsinki, where we saw the first N700 devices in the hands of some lucky Debian/Nokia people using it as an Internet tablet. N900 has been a strong leap forward for the platform, before its head was teleported sideways by the whole MeeGo merger/debackle. Again I have two options here: Get a N900 either here in Latvia for 325 LVL (that would be the total cost over 2 years) or get it from USA during Debconf for 399 USD (+129 USD for 2 year warranty) = 317 LVL. And additional option would be to wait until October/November and then buy the new Nokia MeeGo device, rumoured to be N9-00 and likely to be around the same price as N900 was when it was introduced (around 500 LVL). Benefits: Drawbacks: Option 3 Android After looking trough the options for Android phones, I ve narrowed the selection to the ones that are either available from local carriers or can be easily gotten unlocked also the phones need to have announced plans to have at least Android 2.2 version. Currently the choice is limited to HTC Desire (from carrier or unlocked) and Samsung Galaxy S (unlocked). The prices over 2 years break down as follows: HTC Desire (locked to LMT): 69 + 9*24 = 285 LVL
HTC Desire (locked to Bite): 199 + 2*24 = 247 LVL (Note: only 500Mb of data per month available)
HTC Desire (locked to Tele2): 179 + 6*24 = 323 LVL
Note: all the above options would also require me to pay a 45 LVL early termination fee on my current iPhone contract if I choose to do this before September.
HTC Desire (unlocked): 311 LVL
Samsung Galaxy S (unlocked): 350 LVL (import from Germany) Benefits: Drawbacks: Please correct me if I am wrong with something!!! And also it would be cool if you expressed your own opinions in the comments. Currently I am very undecided about what I am going to do, but after completing this entry I am leaning towards an unlocked HTC Desire. Update:
Olivier Cr te shares a N900 vs. Nexus One experience Edit 01-07-2010: Handed my iPhone in for a warranty repair, got a dumbphone Nokia as a loaner during repairs. Very surprised about how long a battery can last on a low powered and cheap phone. In the mean time from all the comments here and elsewhere I am starting to see that I am too annoyed with the iPhone platform (mostly iTunes) to stay there and that I am also not convinced in the direction MeeGo is going (a lot of community developers are annoyed and are jumping ship), so N900 is also out. I will need to see 2-3 MeeGo smartphones and also see how Nokia will treat their smartphone users and developers on this platform, in the long term, before I ll be ready to trust them with my money. Therefore, my choice is becoming pretty clear look for the best Android phone (after my iPhone dies). We have this thing at Accenture, where we can get company phones at a discount and HTC Desire is on that list. If I can get that, it would slash 100 LVL (almost a third of the price!) off it, but to get there I either need to get a promotion (rather rare after just one year with the company) or prove the necessity of the phone for the needs of my project (kinda hard currently). Well, the situation will be a bit clearer next week when I get my iPhone back. (Apparently the Latvia s largest holiday Midsummer Festivities or J i/L go with tons of traditional outdoor activities, caused a lot of phones to become broken and there is a backlog of warranty service :) ) So at this point my plan is: get my iPhone repaired and get a promotion or a project where I can justify a company-paid HTC Desire and then hope that Nokia/Intel really get their stuff together and make MeeGo a great smartphone platform over the next year or two. Update: Got my iPhone 3G back from the warranty . It is in quotes because they just gave me a new iPhone 3G. The battery lasts for 3 days of minimal use and it has not crashed yet. The recovery process was a pain again caused by iTunes: iTunes refused to restore my backup onto the new phone, because my backup was made on a iOS 4.0, while the new phone I got from the warranty has 3.1.3 on it. So I had to set this phone as a new phone, initialise it, make a backup, upgrade to 4.0, restore from the (empty) backup, then reset the phone to factory settings again and only then I could start restoring data from my original (pre-warranty) backup and start copying my data on to the device. All in all it took nearly 3 hours if iTunes doing something and almost a dozen reboots of the phone. Almost half of that time was spent doing completely unnecessary steps to work around the fact that iTunes is braindead. I ll continue using my iPhone for now, keeping my eyes on the newest Android phones and also waiting for the long-promised first MeeGo phone from Nokia.

12 June 2010

Russell Coker: Links June 2010

Seth Berkley gave an interesting TED talk about developing vaccines against the HIV and Influenza viruses [1]. The part I found most interesting was the description of how vaccines against viruses are currently developed using eggs and how they plan to use bacteria instead for faster and cheaper production. One of the problems with using eggs is that if the chickens catch the disease and die then you can t make a vaccine. Aigars Mahinovs wrote a really good review of Microsoft Azure and compared it to Amazon EC2 [2]. It didn t surprise me that Azure compared poorly to the competition. Johanna Blakley gave an insightful TED talk about IP lessons from the fashion industry [3]. She explained how an entire lack of IP protection other than trademark law was an essential part of the success of the fashion industry. She also compared the profits in various industries and showed that industries with little or no IP protection involve vastly larger amounts of money than industries with strong IP protection. Lisa D wrote an insightful post about whether Autism Spectrum Disorders (such as Asperger Syndrome) should be considered to be disabilities [4]. I don t entirely agree with her, but she makes some really good points. Sharmeen Obaid-Chinoy gave an interesting TED talk about the way the Taliban train young children to become suicide bombers [5]. Apparently the Taliban prey on large poor families, sometimes paying the parents for taking children away to school . At the Taliban schools the children are beaten, treated poorly, and taught theology by liars who will say whatever it takes to get a result. Then after being brain-washed they are sent out to die. Wired has an interesting article about Charles Komanoff s research into New York traffic problems [6]. He aims to track all the economic externalities of traffic patterns and determine incentives to encourage people to do things that impose less costs on the general economy. His suggestions include making all bus travel free as the externality of the time spent collecting fares is greater than the fare revenue. It s a really interesting article, his research methods should be implemented when analysing traffic in all large cities, and many of his solutions can be implemented right now without further analysis such as free buses and variable ticket pricing according to the time of day. William Li gave an interesting TED talk about starving cancer by preventing new blood vessels from growing to feed it [7]. Drugs to do this have been shown to increase the life expectancy of cancer patients by more than 100% on average. Also autopsies of people who died in car accidents show that half the women in their 40 s had breast cancer and half the men in their 50 s prostate cancer but those cancers didn t grow due to a natural lack of blood supply, so the aim here is to merely promote what naturally happens in terms of regulating cancers and preventing them from growing larger than 0.5mm^3. There are a number of foods that prevent blood vessels growing to cancers which includes dark chocolate! ;) Also drugs which prevent blood vessel growth also prevent obesity, I always thought that eating chocolate all the time prevented me from getting fat due to the central nervous system stimulants that kept me active Graham Hill gave an inspiring TED talk about becoming a weekday vegetarian [8]. Instead of making a commitment to being always vegetarian he s just mostly vegetarian (only eating meat on Sundays). He saves most of the environmental cost and doesn t feel guilty if he ever misses a day. It s an interesting concept. Cory Doctorow wrote an insightful article for the Guardian about the phrase Information Wants To Be Free [9]. He points out that really it s people who want to be free from the tyranny that is being imposed on us in the name of anti-piracy measures. He also points out that it s a useful straw-man for the MAFIAA to use when claiming that we are all pirates. The Atlantic has an interesting article about the way that Google is working to save journalistic news [10]. Adam Sadowsky gave an interesting TED talk about creating a Rube Goldberg machine for the OK Go video This Too Shall Pass [11]. At the end of the talk they include a 640*480 resolution copy of the music video. Brian Cox gave an interesting TED talk advocating increased government spending on scientific research [12]. Among other things he pointed out that the best research indicates that the amount of money the US government invested in the Apollo program was returned 14* to the US economy due to exports of new American products that were based on that research. It s surprising that any justification other than the return on investment for the Apollo program is needed! Moot gave an interesting TED talk about Anonymity [13]. I don t think that he made a good case for anonymity, he cited one person being identified and arrested for animal cruelty due to the efforts of 4chan people and also the campaign against the Cult of Scientology (which has not been very successful so far). Rory Sutherland gave an intriguing TED talk titled Sweat the Small Stuff [14]. He describes how small cheap changes can often provide greater benefits than huge expensive changes and advocates corporations having a Chief Detail Officer to take charge of identifying and implementing such changes. TED Hosted an interesting debate between pro and anti nuclear campaigners [15]. They agreed that global warming is a significant imminent problem but disagree on what methods should be implemented to solve it.

20 May 2010

Aigars Mahinovs: So, what is Microsoft Azure and how it compares to Amazon s AWS

We had a Microsoft salesperson stop at our offices today to tell people about wonders of cloud computing and Microsoft s Azure will save us all. The following is a short and non-exhaustive list of what Azure does not do according to their own experts: The Microsoft presenter also repeatedly engaged in shady tactics when presenting his information: So to summarize, Microsoft has build a cloud of Windows-like machines which can run your programs (as long as they are in .Net or Silverlight) and which does not have even a third of the features of such de-facto market leader as Amazon AWS. They try giving away their product for free and only get 14k applications in 6+ months. Note: Amazon has at least 300 000 public applications and God only knows how many more private ones. They will grow, due to companies locked-into Microsoft s way of thinking buying into their marketing, but if you have a head on your shoulders, you should avoid it as a trap, which it actually is. All that can be done on Azure, can also be done on Amazon. The opposite is not always true. In addition, if you will want have a mixed environment with both Windows and Linux servers, you will not be able to deploy Linux servers to Azure (or they will be very slow) and you ll have to put your Linux servers into Amazon (or another) cloud and pay double traffic fees for every byte of traffic between your own servers. Microsoft Azure prices are exactly the same as Amazon EC2 prices with Windows instances. However on Amazon you actually get fully capable Windows Server instances. And if you don t need Windows, you can get a much better price on Amazon. The only companies that can safely use Azure are companies that only have Windows servers AND plan to only have Windows servers for the foreseeable future. Knowing how dominant Linux is in the server market and how many Windows-only shops actually have a few Linux servers in the back room, there is very little future for Azure, until they wise up and make Linux instances first class citizens in their cloud. If not because their customers need them, then because their customers think that they at least might need them in the future. Update: Amazon now released a Reduced Redundancy storage for S3 that provides the exacts same level of storage guarantees that regular Microsoft Azure SLA (99.9%) for two thirds of the price, and also Amazons now explicitly says that their regular S3 is Designed for 99.999999999% Durability , which is worlds above what Microsoft provides. Now in this case Amazon again is cheaper than MS. Also Amazon bandwidth prices are 33% lower compared to Azure. In Azia the difference is simply staggering (0.45$ vs 0.19$ per Gb).

13 May 2010

Aigars Mahinovs: Hacker Neo caught in Latvia

A scandal has been brewing in Latvia over the last half year and yesterday the activity spiked shocking the media and some IT people in the country. I ll go back and explain what happened first, what is happening now and why this could have a heavy impact on IT and journalists in Latvia. At the end of last year, there were rumours that the IT system of Latvia s Internal Revenue System was hacked and millions of documents had been downloaded by multiple organizations. Shortly thereafter more details on the glaring security hole became public (after it was closed). There is a full electronic interface to give all reports to the IRS electronically (at http://eds.vid.gov.lv) and as part of that system you could also view and export monthly report summaries about your organization into XML and PDF files. After the system checked that you are authorized to access the report, you were redirected to the URL to actually download the report by report ID (as a single param in a GET request). Unfortunately, report IDs were predicable and the script that gave the reports for download did not check if you were authorized to get that report. It did not even check if were logged into the system. There were suspicions that the authorization was disabled on purpose to allow to leak data on purpose, but apparently it was an error of forgetting to disable debug code in production environment. The error was discovered only because the firewall administrator noticed an unexplained stable increase of traffic, especially during night hours when typically the traffic fully stopped. Apparently a single hacker (who later identified himself as Neo to the press) discovered the flaw and wrote a script to just try all possible report ids and get as much data out as possible. This had been going on for months, before someone noticed. After the flaw was discovered and a bit of time passed, Neo made his first move he published the list of top salaries in a governmental company, that clearly showed that the top leadership of this company failed to cut their salary by 40%, like everyone elses during harsh budget cuts of 2009. He stripped the names and ids of the specific employees, but named the company which made it pretty easy to figure out who was who. The society was outraged that the top managers in a government owned company failed to comply with the strict pay cut that everyone else in government had to endure. But after a few weeks the outrage subsided and no action followed from the government or law enforcement. Neo continued to release documents detailing salaries of top managers in different Latvian government companies. And each time after short outrage, nothing happened. Neo gave an interview where he said that he was disappointed in the passivity of the Latvian people in face of such blatant injustices. After a few month Neo went silent, promising to return before parliamentary elections this fall. However, this week a new development shocked everyone in the middle of the night two police SWAT teams went into action: one detained Ilm rs Poik ns, a researcher in artificial intelligence at the University of Latvia s Computer Science department and another raided the home of a Latvian TV journalist Ilze Nagle who interviewed Neo. Poik ns confessed of being Neo the next day and was released (with travel restrictions, pending trial) today. Politicians reacted immediately opposition demanded the resignation of the Interior Minister over such blatant disregard of freedom of press and another politician (who is also a famous lawyer) Aleksejs Loskutovs volunteered to defend Neo pro-bono (on Twitter, no less). Almost all Latvian online media have the arrest of Neo and the raid on the home of a journalist as main stories of the day. As a legal titbit, we also know that Neo is being charged with breaking statutes 145 and and 244p2 of the criminal law. Statute 145 is hard to find applicable in this situation as talks about actions done by people authorized (..) to access [private] information . Statute 244p2 will also be hard to pin down as it mentions influencing system resources of (an IT system) and if such action caused severe harm . It looks like the first part talks about at least a DoS attack (which did not happen in this case) and also there was no measurable harm from these leaks. Also Neo was careful to strip all personally identifying information (such as names, social security numbers and addresses of the employees in question), so it will be hard to pin him on that. Also no actual breaking or other modification of an IT system occurred. And no specialized software was used beyond a trivial script such as :
for i in range(0,7000000):
    wget('https://eds.vid.gov.lv/getRep.aspx?id='+str(i))
A lot of commentators on the Internet likened the situation to walking trough an unlocked door and stealing something. I think that analogy is very incorrect there was no door, and nothing went missing after the action. I came up with a different analogy there was this corridor with a lot of doors in IRS, locked steel doors. You were instructed to go to a room with a specified number and given a key to that room to unlock it and see your secret info. However, that corridor opened out to the street on one end, oh and also the walls of the rooms with all the secrets were transparent. So Neo walked into the corridor, looked at some of the secrets, wrote them down (to remember them better) and then went out and discussed the worst examples abuses of power he saw. In the end IRS had to learn their lesson if you have to put naked photos of yourself on the Internet (or something equally embarrassing), then make damn sure you password protect that, but if you don t then don t cry that someone hacked you and stole you pictures. What other people think:
http://freespeechlatvia.blogspot.com/2010/05/neo-released-under-restrictions.html We ll see how the story develops soon.

6 May 2010

Aigars Mahinovs: Going to Debconf10!

After a few months, today I finally got the approval and As I work for Accenture now, I got them to pay for the plane tickets and also spring for the Professional registration, so more people can attend. I am very happy, and very grateful to the company and to the managers here in Riga office for being so very supportive. It is refreshing to see that even a large corporation can be quite nimble. I hope I can get on a direct Riga New York flight (there actually is one, once a week, by Uzbekistan Airways) which would be a wicked cool way to save travel time and worries. This might even be faster end-to-end than last year going to Spain (via Frankfurt). Now I only need to replace my passport with the new biometric one, to qualify for USA Visa Waiver and I ll be good to go :)

28 March 2010

Aigars Mahinovs: Ubuntu 10.04 and NTFS filesystems

subj. don t mix just upgraded a simple Ubuntu 9.10 to Ubuntu 10.04 and it failed to boot. After careful examination, it looks that something replaced the munt line of my NTFS partition in the /etc/fstab and claimed that it is a VFAT partition and mountall that is run during boot gets very, very confused if presented with such dillema, so mach in fact that it hangs and stops the whole boot sequence. The workaround is to boot from a livecd/usb and comment out all NTFS and VFAT lines in /etc/fstab. If that still does not help replace the large UUIDs with device names back. Still have not reported the bug as there look to be several regression in NTFS support, the upgrader corrupting the fstab file and mountall incorrectly handling a case of an unmountable file system. P.S. Also my Firefox would not start that was solved by removing the sessionstore* files in my profile.

19 March 2010

Aigars Mahinovs: Latvijas pavasara Ubuntu Bug Jam un Installest 27.03.2010

The following is an invitation to the Latvian Ubuntu Bug Jam (in Latvian) sent for a bit of a wider circulation to catch people that monitor Planet Debian, but not Planet Ubuntu.lv. 27. mart LU Linux centr notiks divi pas kumi vien Ubuntu Global Bug Jam Latvijas da a un installfests. Global Bug Jam ir pas kums, kur piedal ties ir aicin ti interesenti, speci listi, studenti, lai mekl tu k das Ubuntu Lucid Lynx test anas versij . Cilv ki, kas grib uzzin t par Ubuntu Linux, vai kuri grib atrisin t k du konkr tu probl mu ar Ubuntu Linux tiek aicin ti n kt uz pas kuma otro da u no pulksten 14:00 l dz 16:00. Pas kuma b s kafija un bulci as ar Accenture atbalstu. Ubuntu Global Bug Jam ir glob ls pas kums, kura m r is ir iepaz stin t programm t jus un tulkot jus ar r kiem, kas tiek lietoti, lai labotu probl mas Ubuntu oper t jsist m un ar izlabot p c iesp jas liel ku skaitu probl mu s laik . Izstr d t ji, kas grib labot Ubuntu probl mas vai iem c ties k labot Ubuntu probl mas, tiek aicin ti ierasties 12:00 un palikt l dz 16:00. Installfest pas kuma sada tiek aicin ti visi eso ie Ubuntu lietotaji, kuriem ir k das konkr tas probl mas un ar cilv ki, kas tikai v l interes jas par Ubuntu Linux. Ja jums ir konkr ta probl ma ar Ubuntu Linux ir ieteicams atnest uz pas kumu savu datoru, kur o probl mu var atk rtot, lai pas kum eso ie programm t ji var tu noteikt s probl mas iemeslu un pal dz tu no nov rst. Installfests s ksies 14:00 un turpin sies l dz 16:00. Ubuntu ir bezmaksas, uz Linux balst ta pilna apjoma oper t jsist ma jebkuram person lajam datoram, serverim un portat vajai iek rtai. T s standarta komplekt iek autas visas nepiecie am s programmas, lai str d tu ar tekstiem, att liem, elektronisko pastu un Internetu, k ar j s varat instal t papildus programmat ru da diem nol kiem. Pasaul to obr d jau lieto vair k k 8 miljoni cilv ku, un to leg li bez maksas var lietot gan m j s, gan komerci l s un nekomerci l s organiz cij s. LU Linux Centrs ir izveidots Latvijas Universit tes Datorikas fakult t . Linux Centra darb bas m r i ir: populariz t atv rt pirmkoda (Open Source) programmat ras, tai skait , Linux oper t jsist mas un citu atv rto tehnolo iju iesp jas un priek roc bas; piedal ties LU studiju proces un stenot lieti o IKT p t jumu projektus, tajos izmantojot un att stot atv rt s tehnolo ijas; sekm t APP pieejam bu Latvij un pasaul .

7 March 2010

Aigars Mahinovs: Linuksys WRT54GL

Daniel, I also recently installed a WRT on the Linksys WRT54GL. But I used DD-WRT instructions went to http://www.dd-wrt.com/site/support/router-database, entered my model number and got direct download links to the firmware along with a README. One very important points in flashing these routers is to clear the NVRAM by doing a hard reset BEFORE and AFTER an upgrade to a different type of software on the device. Also, starting with a micro build is strongly recommended. In any case the hardware looks to be very solid, especially if you don t need gigabit ethernet and n wireless, but might need some advanced networking features.

10 February 2010

Aigars Mahinovs: USB 3D Sound

Want a sound card? Want a cleared sound that is not contaminated by electromagnetic interference in you computer s case? Want that all to work in Linux? I did. So I got me one of theseUSB sound cards. It arrived today in a tiny padded envelope. USB sound card And it works with Linux. Just plug and play. PulseAudio makes it all trivial. The device ID as reported by lsusb is 0c76:1607 JMTek, LLC. The text on the back of the device says Model HY544 USB 3D Sound Pnp FC CE Made in China .

9 February 2010

Aigars Mahinovs: Debug and optimization do NOT mix!

This has robbed me of several days of my life, so I want to bring Google juice this this problem. IF you have a Pylons or TurboGears application or anything else that uses the fantastic EvalException WSGI middleware for web debugging of you web program and have the following symptom: * on a crash the traceback page shows up, but it has no css style and no images
* http://127.0.0.1:5000/_debug/media/plus.jpg returns a 404 (where 127.0.0.1:5000 is the path of your application) with Nothing could be found to match _debug The problem that you are facing in this case is the enviroment variable PYTHONOPTIMIZE or apache_wsgi option WSGIPythonOptimize. Unset them both or you debug environment will not work!

19 January 2010

Aigars Mahinovs: /bin/time 0maxresident?

If I run GNU time with some large program, I get the following output:
604.90user 13.16system 11:05.56elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k
712inputs+0outputs (5major+76648minor)pagefaults 0swaps
Question I would guess that maxresident would refer to maximum resident memory usage of the program that I ran, but why is it always zero?

29 November 2009

Aigars Mahinovs: New hardware planning

My primary workstation is a 3 and a half year old Dell XPS M1710 laptop and it is getting old the 320 Gb hard drive is getting small and slow, the 3 Gb or RAM (expanded from 2 Gb) look too small and screen is turning brown in one corner. Also dead or dying: keyboard (after cat+coffee incident), built-in speakers, battery, power adapter (twice replaced), DVD writer (rads but does not write any more) and fans (one replaced, one getting louder by the week). Also the video card is a bit slow for nowadays needs. So I set aside some money and started considering the options. In short, I could get another laptop OR get a regular computer for the same price but with twice the power. After considering all the laptop options up to 1000LVL, I decided to get a regular computer and if/when I ll need to travel I ll either take one of the old laptops with me or get a new netbook. Now comes the fun part choosing the components! So far my choices are as follows: And that is it a great setup that will last me a few years for around 700 LVL and it will have more partial upgradability than my laptop that I paid twice as much 3 years ago. Any tips? Did I miss anything? Is it likely that anything in the above list will cause me any trouble in latest Ubuntu/Debian installations? Update: got all of the above, except 500 Gb HDD of the same kind (faster, cooler, but higher price per Gb) and no Blueray (no drive in LV at the moment and did not want to wait a few weeks). The results are good, but I wish I would have gone with a 5 LVL more expensive video card that has a quieter cooler. Now I am buying a separate video card cooler for around 20 LVL that will be both more efficient and much quieter than the stock cooler I have Zalman VF1000 is the model. Has not arrived yet. P.S. It looks like updating a post in WP, automatically bumps it back into the Planet feed. Sorry :(

1 November 2009

Aigars Mahinovs: Gnome typing break has no way to lock the screen

Want a definition of a paper cut bug? Here it is. And here and here are two more side effects of the same bug. The original bug report will be 6 years old in a month. Can we do something to prevent this bug surviving that long?

24 October 2009

Aigars Mahinovs: QEMU quit during console startup bind() failed

To get more Google juice to the problem. If you are trying qemu or libvirt or kvm or virt-manager and when you are starting up your second guest you see a message such like this:
QEMU quit during console startup
bind() failed
Then two of your guests have the same port set for their VNC devices. Change the port in the VNC section to something that is free. A more specific erro would have save a half an hour of Googling.

Next.

Previous.